Object Recognition with and without Objects

While recent deep neural network models have given promising performance on object recognition, they rely implicitly on the visual contents of the whole image. In this paper, we train deep neural networks on the foreground (object) and background (context) regions of images respectively. Considering human recognition in the same situations, networks trained on pure background without objects achieves highly reasonable recognition performance that beats humans to a large margin if only given context. However, humans still outperform networks with pure object available, which indicates networks and human beings have different mechanisms in understanding an image. Furthermore, we straightforwardly combine multiple trained networks to explore the different visual clues learned by different networks. Experiments show that useful visual hints can be learned separately and then combined to achieve higher performance, which confirms the advantages of the proposed framework.